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TIGHT INEQUALITIES AMONG SET HITTING TIMES IN 

MARKOV CHAINS 

SIMON GRIFFITHS, ROSS J. KANG, ROBERTO IMBUZEIRO OLIVEIRA, 
AND VIRESH PATEL 

Abstract. Given an irreducible discrete-time Markov chain on a finite state 
space, we consider the largest expected hitting time T(a) of a set of stationary 
measure at least a for a £ (0, 1). We obtain tight inequalities among the 
values of T(a) for different choices of a. One consequence is that T(a) < 
T(l/2)/« for all a < 1/2. As a corollary we have that, if the chain is lazy in 
a certain sense as well as reversible, then T(l/2) is equivalent to the chain's 
mixing time, answering a question of Peres. We furthermore demonstrate 
that the inequalities we establish give an almost everywhere pointwise limiting 
characterisation of possible hitting time functions T(a) over the domain a £ 
(0,1/2]. 



1. Introduction 

Hitting times are a classical topic in the theory of finite Markov chains, with 
connections to mixing times, cover times and electrical network representations [5J 
[6] . In this paper, we consider a natural family of extremal problems for maximum 
expected hitting times. In contrast to most earlier work on hitting times that 
considered the maximum expected hitting times of individual states, we focus on 
hitting sets of states of at least a given stationary measure. Informally, we are 
interested in the following basic question: how much more difficult is it to hit 
a smaller set than a larger one? (We note that other, quite different extremal 
problems about hitting times have been considered, e.g. [3]) 

Following the notation of Levin, Peres and Wilmer [5], we let a sequence of 
random v.ariables X = (Xi)%l. denote an irreducible Markov chain with finite 
state space fi, transition matrix P, and stationary distribution n. We denote by 
Ho some initial distribution of the chain and by P Mo the corresponding law. In the 
case that /j,q = x almost surely, for some i £ SJ, we write ¥ x for the corresponding 
law. 
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Given a subset A C f2, the hitting time of A is the random variable ta defined 
as follows: 

t a = min{ t : X t 6 A } . 

We shall take particular interest in the maximum expected hitting times of sets of 
at least a given size. For a S (0, 1) we define T(a) — T p (a) as follows: 

T(a) = max{E a! [ta] : x £ n, A C fi, vr(A) > a } . 

In other words, T(a) = T p (a) is the maximum, over all starting states Xq = x 6 f2 
and all sets ^4 C f2 of stationary measure at least a, of the expected hitting time 
of A from x. 

1.1. The extremal ratio problem. Note the obvious fact that, given < a < 
6 < 1, T(a) is lower bounded by T(/3) always. Informally in other words, it is 
more difficult to hit smaller subsets of the state space. A natural problem then is 
to determine how much more difficult this is, i.e. how large the ratio between T(a) 
and T(J3) can become. We dub this the extremal ratio problem. 

Problem 1.1. Given < a < f3 < 1, what is the largest possible value of 
T(a)/T(0) over all irreducible finite Markov chains (on at least two states)? 

A first result on this problem was noted by the third author [7J Corollary 1.7]. 

Theorem 1.2. Fix < a < < 1/2. There exists a constant Cp > such that 
the following holds. For any irreducible finite Markov chain, 

1 (a) < Gfl • . 

a 

This can be shown via Cesaro mixing time, specifically as a consequence of an 
equivalence between T{0) for j3 £ (0,1/2) and Cesaro mixing time for any irre- 
ducible chain. This equivalence, which was recently proved independently by the 
third author [7] and by Peres and Sousi [5], we will discuss in more detail in Sub- 
section 11.31 

In this paper, we improve upon the above result significantly, without recourse to 
any results on mixing time. Our first main result implies that the optimal constant 
in Theorem 11.21 is Cp = 1 and that moreover we can include the case (3 = 1/2. 

Theorem 1.3. Fix < a < (3 < 1/2. For any irreducible finite Markov chain, 

(*) T(a) < T([3) + (l-l).T(l-P) < ^M. 

This bound on T(a) is tight: for any < a < /3 < 1/2, there exists an irreducible 
finite Markov chain for which the three terms in Q are all equal. Furthermore, 
3 = 1/2 represents a boundary case for Theorem 11.31 for each > 1/2, there is a 
class of irreducible finite Markov chains such that T(a)/T(/3) is arbitrarily large. 
Thus we have completely settled the extremal ratio problem. 

As an application of Theorem ll.3[ we show in Subsection 1 1 . 31 how mixing time is 
equivalent to T(l/2) for any irreducible chain, under the added restriction that the 
chain is lazy in a certain sense as well as reversible; this resolves a problem posed 
by Peres [4]. 
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Our strategy for proving Theorem 11.31 relies on a simple, but useful proposition, 
which can be deduced from the ergodic properties of irreducible finite Markov 
chains. We require the following definitions. Given two sets A, B C f2, we define 

d + (A, B) = maxE x [t b ] and d" (A, B) = minEa, W B ] . 

Proposition 1.4. Given an irreducible Markov chain with finite state space Q and 
stationary distribution n, let A, C C fl. Then 



Both Theorem 11.31 and Proposition 11.41 are proved in Section [5] The examples 
mentioned after the statement of Theorem 11.31 are presented in Section [3] We 
remark that we have replaced our original proof of Proposition ll.4l bv a shorter and 
more elegant argument of Peres and Sousi [TU] . 

1.2. The shape problem. In consideration of Theorem ll.31 it is natural to wonder 
what form the ratio T(a)/T(j3) may possibly take. The second problem we treat 
is what we call the shape problem. 

Problem 1.5. What is the minimal set of constraints on the possible "shape" of the 
function T(a) over the domain a G (0,1/2] over irreducible finite Markov chains 
(on at least two states)? 

We show that, in the appropriate limit, the constraints imposed by Q in Theo- 
rem [T3] are the only non-trivial constraints on T{a) over the domain a G (0, 1/2]. 
(The trivial constraint is that T must be a decreasing function.) 

We now make this statement rigorous. Let T denote the set of decreasing func- 
tions / : (0,1/2] -4 K given by f(a) — T(a)/T(l/2) for some irreducible finite 
Markov chain (on at least two states). We also consider limits of such functions. 
Let T denote the set of decreasing functions / : (0, 1/2] — > M each of which may 
be obtained as the almost everywhere (a.e.) pointwise limit of functions in T . Our 
second main result is as follows. 



Theorem 1.6. Let f : (0, 1/2] — > R be a decreasing function. Then f G T if and 
only if f (1/2) = 1 and 



We prove this by way of a class of chains we call L-shaped Markov chains, for 
which the hitting time functions T(a) can be straightforwardly determined. We 
show Theorem 11.61 in Section [3] 

As it turns out, the constraints given by (j*j) for < a < (3 < 1/2 are not the only 
non-trivial constraints on T(a) over the larger domain a G (0, 1). We demonstrate 
this in Section^ The shape problem over that larger domain remains an interesting 
open problem. 

1.3. The connection to mixing times. To put our results into wider context, 
we now describe the relationship between Theorem 11.31 and mixing times. Recall 
that the (standard) mixing time of a chain with state space f2, transition matrix 
P, and stationary distribution tt is defined as 



tt(A) < 



d+{A, C) 



d+(A,C)+d~(C,A) ' 



/(«) < - 
a 



for alia G (0,1/2). 
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This parameter has various connections to the analysis of MCMC algorithms, to 
phase transitions in statistical mechanics, and to other pure and applied prob- 
lems [5] . Aldous [T] showed that it is also related to other parameters of the chain, 
including the following hitting time parameter: 

t P rod = max{n(A)E x [ta] : x € SI, + A C SI}. 

Theorem 1.7. There exists a universal constant C > such that the following 
holds. Consider a reversible, irreducible finite Markov chain with transition matrix 
P that is lazy in the sense that P xx > 1/2 for all x in the state space. Then 

f P 

mix j-P j-P 
q — ''prod — ''mix- 

We remark that Aldous proved Theorem 11.71 in continuous time, but there are 
standard methods to transfer his result to discrete time (cf. [SJ Theorem 20.3]). 

Aldous's theorem is typically summed up by saying that i mix and i P rod are "equiv- 
alent up to universal constants" , or simply "equivalent" . A similar equivalence was 
proved for all irreducible finite Markov chains (not necessarily lazy or reversible), 
with i p ix replaced by Cesaro mixing time [2]: 



t£ cs = min \ t £ N : Vi e O, VA C Si, 



1 * _1 



~ 4 



s=0 

A drawback of Theorem 1 1 . 71 and its Cesaro mixing version is that it might seem 
that the mixing time depends on the hitting times of arbitrarily small sets. On the 
contrary, it transpires that the maximum hitting times of only sets that are large 
enough is also equivalent to i mix and i p es (in the analogous senses). The following 
was proved independently by Peres and Sousi [5] and by the third author [7]. 

Theorem 1.8. For each a € (0,1/2), there exists a constant c(a) > such that 
the following holds. Consider a reversible, irreducible finite Markov chain with 
transition matrix P that is lazy in the sense that P xx > 1/2 for all x in the state 
space. Then 

^<T p (a)<c( ft )C. 
c(pt) 

Moreover, for any irreducible finite Markov chain (not necessarily reversible or 
lazy), 

^<T p (a)<c(a)t^ s . 

Note that, together with the Cesaro mixing time form of Theorem 1 1.7[ Theorem ll.2l 
now follows. 

There is no analogue of Theorem 11.81 if one allows a > 1/2: a simple counter- 
example is given by a random walk on a graph consisting of two large cliques 
connected by a single edge [SJ. Until now, it was not known whether T p (l/2) is 
also equivalent to t^ ix and iQ es . We prove here that this is the case, answering a 
question of Peres [3] . 

Theorem 1.9. There exists a universal constant c > such that the following 
holds. Consider a reversible, irreducible finite Markov chain with transition matrix 
P that is lazy in the sense that P xx > 1/2 for all x in the state space. Then 



^ < T p (l/2) < ct p 



mix ' 
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Figure 1 . An illustration of the situation in Theorem 11.31 

Moreover, for any irreducible finite Markov chain (not necessarily reversible or 
lazy), 

%<T P (1/2)< C ^ CS . 

Proof. By Theorem 11.71 and its Cesaro mixing time version, it suffices to show that 
^prod i s equivalent to T p (l/2). But this is simple: on the one hand, 

T p (l/2) 

< max{Tr(A)E x [t a ] : x € ft, A c ft, it (A) > 1/2} < t P Tod , 

whereas, on the other hand, Theorem 11.31 implies that 

n(A)E x [r A ] < n{A)T p ^{A)) < T p (l/2) 

if ir(A) < 1/2, and the fact that T p (-) is monotone decreasing implies the above 
inequality also holds if 7t(j4) > 1/2. □ 

1.4. Organization. The remainder of the article is organised as follows. In Sec- 
tion [2J we prove Theorem 11.31 In Section |31 we show Theorem 11.31 is tight by 
presenting some two- and three-state Markov chains. We also prove Theorem 11.61 
in Section[3J Finally, in Section@]we consider the behaviour of T(ot) over the larger 
domain a 6 (0, 1) and make some concluding remarks. 

2. Proofs for Theorem 11.31 

We begin by showing that Theorem ll.3l is an easy consequence of Proposition ll.4l 

Proof of Theorem \1.3[ Consider an irreducible Markov chain with finite state space 
ft and stationary distribution n. Fix a state x 6 ft and a set A C ft with 7r(yl) > a. 
We prove that 

Ex [ta] < W) + Q- - lj ■ T(l - 0) . 

Since x and A are arbitrary, this will suffice to prove the theorem. 
Define the set C = C^ A as follows: 

C = ly e ft : E y { TA ) > - l) • T(l - f3) 

We claim that ir(C) < 1 — /3. Indeed, if, on the contrary, 7r(C) were at least 1 — j3, 
then it would follow that d+(A,C) <T(l-/3) while d~(C,A) > (a" 1 - 1)T(1 - ji). 
This would imply, by Proposition ll.4[ that n(A) < a, a contradiction. Thus, letting 
B = ft \ C, we have established that tt(B) > (3. Our route from x to A is now clear 
- proceed from x to B and then on from B to A. See Figure [U That is, using 
the Markovian property of the chain, the expected hitting time of A from x may 
be bounded by 

Ex[ta] < E x [tb] + d+(B,A). 
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Combining the bound E^ [t b ] < T{fi) (since tt(B) > /3) with the bound d + (B, A) < 
(a^ 1 — 1) • T(l — j3) (since B is the complement of C), we obtain 

E* [t a ] < T{p) + (~ - lj ■ T(l - p) , 

as required. □ 

All that remains is to prove Proposition 11.41 As remarked in the introduc- 
tion, we have replaced our original proof by a shorter and more elegant argument 
suggested by Peres and Sousi |10j . Our original proof, which may be obtained 
at |http: //arxivT org/abs/1209 . 0039vl, relied on the ergodic theorem for irre- 
ducible Markov chains combined with a martingale concentration inequality. 

Proof of Proposition \Ll\ [TO]. Denote the Markov chain by X. Our approach is to 
define a distribution /ion4 and a distribution v on C such that 

(2.1) n(A)E„ [t a ] < (1 - tt(A))E m [t c ] • 

Doing so will complete a proof of the proposition. Indeed, re-arranging inequal- 
ity (|2.1[) , we obtain 

Mtc] / d+(A,C) 



7T(A) < : f L " J ■ : < 

v / — tit r i i tit r_ 1 — 



E^[tc}+E u [t a } ~ d+{A,C) + d-{C,A)' 
as required. 

We now define the distributions /i and v to satisfy inequality (|2. 1[) . Consider an 
auxiliary Markov chain on A defined by the following transitions: for each x, y G A, 
let Q xy be the probability that, started from x, the first state of A hit by X after 
time tc is y (i.e. that y is the first state of A hit after the original chain has 
reached C from x). Let /i denote a stationary distribution of this new chain, and 
let v be the hitting distribution on C when the original chain is started from /x, 
i.e. v{y) = P M (Xrc = y) for each y G C. 

It remains to prove that (|2.1j) holds for this choice of and ia First observe that, 
started from the distribution fi, the expected time the chain X spends in A before it 
reaches C and returns to A is given by E M [r] 7r(A), where r denotes the number of 
steps in such a cycle (from A to C then back to A). This observation is not difficult 
to verify, but we have included a proof below in Lemma lA.ll of the appendix. Next, 
since all visits to A occur before the chain reaches C, we have that E^ [r] it{A) < 
E M [re]. Finally, inequality ([2~T]) follows since E p [r] = E p [t c ] + E, y [ta]. □ 

3. Examples and a proof of Theorem 11.61 

This section is devoted to exhibiting classes of Markov chains which demonstrate 
that Theorem II .31 is tight, in a few different senses. 

We first show that equality in is attained. For each < a < < 1/2 we 
exhibit an irreducible three-state chain with T(a) — T{f3)/a and hence T(a) = 
T(J3) + (a- 1 - l)T{(3) > T(j3) + (c^ 1 - 1)T(1 - /3), as required. Consider the 
three-state chain with transition matrix 

( 1 \ 

e i a+£ a 

(1-a-e) (l-a-e) (l-a—e) J 

V o 1 o J 
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where < e < j3 — a. We note immediately that (e, 1 — a — e, a) is the stationary 
distribution of the chain. It can be easily checked that T(f3) = 1 and T(a) = 1/a. 

We next show that the condition j3 < 1/2 in Theorem ll.3l is necessary by writing 
down an irreducible finite chain with T(f3) = and T(a) arbitrarily large when 
f3 > 1/2. Supposing (3 > 1/2, let N be an arbitrarily large number and let 7 
be such that max{a, 1/2} < 7 < f3. Consider the two-state Markov chain with 
transition matrix 

A-^r ^ y 

\(i-7)v 1 (l-f)Nj 

The stationary distribution of the chain is (7, 1 — 7). It is an exercise to verify that 
T(/3) = and T(a) > (1 - j)N, as desired. 

We now turn to the proof of Theorem 11.61 We must prove that each decreasing 
function / : (0, 1/2] — > M satisfying 

f(a) < - for all a E (0,1/2) 
a 

may be obtained as the a.e. pointwise limit of a sequence of functions /1, /2, • • • in 
T (i.e. functions /; such that fi(a) = T Pi (a)/T Pi (l/2) for some irreducible finite 
Markov chain with transition matrix Pj). We first prove this for a certain class of 
step functions. Then we consider general functions as limits of these step functions 
in order to obtain the theorem. 

The class of decreasing step functions / : (0, 1/2] -4lwe consider are those that 
may be written in the form 

k 

f(a) = 1 + A.; ■ l a < at -, 

i=l 

where the Xi and cti are positive reals satisfying 

i 

(3.1) X^J'- ^ 1 " 1 for each i G {1, . . . , fc}, 

j'=i 

and < ak < • ■ ■ < ol\ < 1/2. We call such a step function hittable. We note that 
if / is a hittable step function then /(1/2) = 1 and f(a) < 1/a for all a £ (0, 1/2). 

Given a hittable step function f(a) = 1 + \ ' la<a; 1 w e define the e-error 

set for / to be the set 

k 

Err/(e) = (J[ai,ai + e], 

i=0 

where we interpret «o = 0. 

Lemma 3.1. Let f : (0,1/2] — 5- K be a hittable step function and e > 0. Then 
there exists an irreducible finite Markov chain such that f(a) — T(a)/T(l/2) for 
allae (0, 1/2] \ Err/ (e). 

The examples of Markov chains we shall use in the proof of the lemma are all 
of the same type. An L-shaped Markov chain is a chain whose state space may 
be labelled = t>o, «i, ■ ■ • ,Vk} in such a way that the transition matrix of 

the chain has non-zero entries only at (i-i)i Piii Pio f° r * ^ {0; !)•••> 

Note that vq is the only state that may be reached directly from a non- adjacent 
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Figure 2. A depiction of an L-shaped Markov chain. 



state. Thus, with the exception of jumps to vq, all transitions are to a neighbour in 
the sequence v-i,Vo,Vi, ■ ■ ■ ,Vk- See Figure [2] In proving Lemma [3TT1 we need only 
consider L-shaped chains. Indeed, it is because the hitting times of such Markov 
chains are relatively easy to determine that they are suitable for our purposes. The 
following lemma, though somewhat specialised, is exactly what we shall require in 
our proof of Lemma |3. II 

Lemma 3.2. Suppose we are given an L-shaped Markov chain on state space Q = 
{V-i,Vq,Vi, . . . ,Vk} with the property that E v . [t Vo ] is maximised at j = —1. If 
i G {0, . . . , fc} and a G (0, 1) satisfy 

(3.2) n({v i+1 , . . . , v k }) + 7r(u_i) < a < ^({u,, . . . , v k }), 

then 

T(a) = E„_! [t^,...,^}] = E v _ 1 [r Vi ] . 

Proof. The second equality is obvious since, starting from the chain first arrives 
in the set {«»,...,«*} at Vi. It is also immediate that T(a) > E„_ 1 [t/ 04) .. ., Vk }] > 
by the definition of T(a) and the assumption that 7r({wj, . . . ,Vk}) > a. Thus 
all that remains is to prove, for any state Vj and set A with tt(A) > a, that 
E Vj [t a ] < E v _, [t { 

Vi,...,V k }\ ■ 

Fix j E {— 1, . . . , k} and a set A with tt(A) > a. Let i' be the minimal non- 
negative integer for which iv £ A. The condition on a implies that i' < i. Now 
(using the property that E Vj [t Vo ] is maximised at j = —1, and the fact that i' < i) 
we have that 

E Vj [t a ] < E Vj [t Vo ] + K Vg [Ty.,] < E„_ 1 [t Vq ] + E Vo [r„J . 

Since any path from v-\ to Vi necessarily passes through vq, the final expression is 
equal to E V l Ws v ^,,. jVk \ \ , completing the proof. □ 

The intuition of the above lemma (at least for our intended application) is that 
if V-i has a very small measure (e say) then for almost all values of a (except on a 
set of measure at most ke) we know how to express T(a) directly as a hitting time. 
This is central to our proof of Lemma 13.11 

Proof of Lemma \3.1[ We shall prove the following assertion: for every hittable step 
function f(a) = 1 + Sj=i ■ la<ai, every < e < 1/2 — oti and every sufficiently 
large natural number N, there exists an L-shaped Markov chain with transition 
matrix P, state space Vt = {u-i, Va,vi, . . . , Vk} and stationary measure tt satisfying 

(i) 7r(w_i) = e, ir(vo) — 1 — ct\ — e, and 7r({w,, . . . , Vf.}) — a>i for each i G 
{l,...,fc}, 

(ii) E Vi [t Vo ] < N for each i G { — 1, 0,1,..., k} with equality if i = —1, and 
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(iii) E Vi _ 1 [t„J = XiN for each i E {1, . . . , k}. 

From this assertion Lemma |3~T1 easily follows. Indeed, since 7r(w ) = 1 — a.\ — e > 
1/2 we have that T(l/2) is precisely the maximum expected hitting time of vq, and 
it follows immediately from (ii) that T(l/2) = N. Given a £ (0,1/2] \Err/(e), 
we shall determine T(a) using Lemma 13.21 and condition (iii). In order to apply 
Lemma 13.21 first notice that condition (ii) ensures that E„. [t„ ] is maximised at 
j = —1. Let i G {1, . . . , k} be smallest such that a < on. Using (i) and the fact 
that a £ (0, 1/2] \ Err/ (e), it is straightforward to verify that (|3.2p holds in the 
statement of Lemma 13.21 Thus, applying Lemma 13.21 and using condition (iii), we 
have 

i I i 

T(a) = E v _ t [ Tvi ] = [t V0 ] +^E„,_ 1 [t Vj ] = 1 + £ A, ] N = f(a)T(l/2), 
as required. 

We now prove the above assertion by stating explicitly the entries of the transi- 
tion matrix P. First, we set 

P-iO = -7T, Po-l = JZ \~at ' -P-l -1 = 1 ^ -P-l 

N (1 — a 1 — e)N 

1 — a% 1 — ca\ — Xia.2 

"oi = 7q \ \ 77) "10 = 7 n \ Ar an d "00 = 1 — "0-1 — "oi- 

(1 — cci — e)X\N («i - a 2 )AiA' 

Next, for each i 6 {2, . . . , fc}, we set 

_ i-Oi(i + Ej= 1 1 A j ) __ l-^q + E^Aj) 

(ai_x - Oi)A(JV ' (a t -a t+1 )A,7V ' 

Pi = -rf and Pjj = 1 - P i0 - Pi - Pi ( i+ i) . 

Last, we set P\\ = 1 — P10 — Pi 2- It is routine to verify that each entry in the 
transition matrix P of our Markov chain is in [0, 1] using (|3.ip . < £ < 1/2 — a%, 
< ctfc < ■ • • < 0:2 < 011, and a large enough choice of N. 

Some straightforward calculations confirm that the resulting stationary distri- 
bution 7r satisfies condition (i) above. Condition (ii) follows easily from checking 
that P i0 >l/N (so that E Vt [t Vo ] < N) for all i and that E v _ 1 [t Vo ] = N. To verify 
condition (iii) for each i 6 {1, . . . , fc}, we compute the expected hitting time from 
Vi—i to Vi by considering the chain started at Vi—i and conditioning on the first 
step. We use induction on i. For the base case (i = 1), we have that 

E^o [t v1 ] = 1 + PooK [t v1 ] + Po-iE^, [t v1 ] 

= 1 + P 00 E.„ [ Tvi ] +P -i(N + E Vo [t Vi ]), 

which implies (after substitution and rearrangement) that E^ [t Vi ] — X\N. Next, 

E V1 [r V2 ] = 1 + Pi iE Vl [t V2 ] + P 10 E VO [t V2 ] 

= l + P xl E Vl [t V2 ] +P 10 {X 1 N + E V1 [t V2 ]), 
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which implies that [t V2 ] = X2N. Finally, for i G {3, . . . , fc}, we have 

[r Vi ] = 1 + P(i-i) (i-2)^ Vi -2 [T Vi } + P(i-x) (i-x)E Cl _i [r Vi ] + P(j_i)oE Vo [r Vi ] 
= 1 + P(i-i)(i-2){^i-iN + E Vi _ 1 [t„J) +P(i_i)(i_i)E t)4 _ 1 [t.„J 
A-i 

+ P(i-i)o X) A ^ + E "*-i 

where the second equality uses the inductive assumption that E v ._ 1 [r„.l = AyiV 
for j € {1, ... ,i — 1}. This implies that K Vi _ 1 [r Vi ] = XiN, as desired. □ 



It is now straightforward to deduce Theorem 11.6 



Proof of Theorem \1.6l The only if part is an immediate consequence of Theo- 
rem [TJ21 Now, fix a decreasing function / : (0,1/2] — > K with /(1/2) = 1 that 
satisfies f{a) < a" 1 for all a G (0, 1/2). Denote by D = £>(/) C (0, 1/2] the set of 
discontinuity points of /. Since / is decreasing, the set D is countable by Froda's 
theorerrQ. For each positive integer n, define the function /„ : (0, 1/2] — » M by 

f n (x)=f(\2 n x]2- n ). 

One easily notes that f n {x) —> f(x) for all x G (0, 1/2] \ D. 

We observe that each f n is a hittable step function, because it can be written 

1+ xa 

1=1 

where a* = 1/2 — «2~™, and A^ = /(a,) — /(a,_i). Condition (|3.1|) is easily seen to 
hold since 

i 

1 + E A J = 1 + ^) - ^ a °) = ^ a i X - 

3 = 1 

To prove the theorem we must find a sequence of functions g n G J- such that 
9n{%) — * f(x) except on a set of measure zero. By Lemma 13-11 there exists for each 
n a function g n G T such that g n (x) = f n (x) for all # G (0, 1/2] \ Errj n (2~ 2n ), 
where 

I I 1 



2n ' 2" 2 2n 



Err / „(2- 2 ")= (J 

8=0 

We now prove that g n (x) — > /(a;) as n — > 00 for each a; G (0, 1/2] \(DU £)'), where 
D' denotes the set of points that lie in infinitely many intervals of Err/ n (2 _2n ). 
Since D U D' has measure zero, this will complete the proof of the theorem. 

To this end, fix x G (0, 1/2] \ (D U £>')• Since 1 £ D, we have that f n (x) -> f(x) 
as n — > 00. Furthermore, since x 2" D' , there exists no such that 

2"- J -l r ■ ■ , 

2 z 1 



2« ' 2« 2 2 ' 1 



u u 

n>no 2=0 

and so g n (x) = f n (x) for all n > n . Thus lim^oo g n (x) = lim^oo = f(x), 

completing the proof of the theorem. □ 



^See http : //en. wikipedia. org/wiki/Froda' s_theorem 
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4. One further result and concluding remarks 

For 0<a</3<l/2we proved the tight inequality T(a) < T(f3) + (a -1 - 
1)T(1 — /?) relating hitting times of large enough sets in irreducible finite Markov 
chains. Furthermore, we demonstrated that this is the only non-trivial restriction 
on T(a) as a function over a G (0, 1/2], in the sense made rigorous in Theorem 1 1.61 

The most obvious remaining question then is whether there are other non-trivial 
inequalities relating the values of T(a) for all a G (0, 1). In one further result, 
we demonstrate that T : (0, 1) — > R is further constrained. However, determining 
the set of all inequalities that hold among the values of T(a) for all a G (0, 1) 
and thereby giving a characterisation in the spirit of Theorem 11.61 of the possible 
behaviour of T : (0, 1) K remains an interesting open problem. 

To demonstrate that T : (0, 1) — ► M is further constrained it suffices to give a 
single example of such an additional restriction, which is as follows. 

Proposition 4.1. Given an irreducible finite Markov chain, suppose T(0.01) = 
99.9T(0.02). Then T(0.99) > 0.1T(0.02). 

We note that this restriction is indeed outside of the class of restrictions imposed 
by Theorem 11.31 Writing T for T(0.02), first one can check using Lemma T3. II that 
there exist Markov chains satisfying the equality T(0.01) = 99. 9T. Furthermore, 
assuming this equality, the application of Theorem 11.31 gives that T(0.01) < T + 
99T(0.98). Although this inequality demands that T(0.98) be very close to T - 
specifically, T(0.98) € [(98.9/99)T, T] — there is no restriction on T(0.99). Thus 
Proposition 14.11 does indeed represent an additional restriction. We require the 
following lemma. 

Lemma 4.2. Given an irreducible Markov chain with finite state space O, let 
A,B,CQQ, and T be a real number such that 

d + (fl : B)<T, d + {n,AuC)<T, d + (fl, A) < 99.9T and d~(B, A) > 98.9T. 
Then d+(B,C) < 14T. 

Proof. Let y G B. Consider running the chain for 10T steps and denote by p y the 
probability V y [ta < 10T). The assumptions on the hitting time of A imply that 

98. 9T < E y [t a ] < 10T + (1 - p y )99.9T. 

Thus p y < 0.111 < 1/8. On the other hand, P y (t AuC < 10T) > 9/10 by Markov's 
inequality, and so P y (r c < 10T) > 9/10 - 1/8 > 3/4. 

We may now bound d + (B, C) as follows. Note that, in the event that the chain 
does not hit C after 10T steps, the expected remaining time to hit C may be 
bounded by T (an upper bound on expected time to return to B) plus d + (B,C) 
(an upper bound on the expected time to hit C from an element of B). Thus 

d + (B,C) < 10T + -(T + d + (B,C)). 

It follows that d+(B, C) < 41T/3 < 14T, as required. □ 

We now prove Proposition 14. II 

Proof of Proposition]^ Let us write T for T(0.02). Since T(0.01) = 99. 9T there 
exists a set A C il with it (A) > 0.01 and a state x G such that E x [ta] = 99. 9T. 
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Figure 3. An illustration of the situation in Proposition l4.il 
Define sets 

B' = {y e Q : E y [t A ] < 99T} and B = {y e Q : E y [t a ] e [98. 9T, 99T]} . 

Arguing as in the proof of Theorem II .31 one obtains that ir{B') > 0.98 — specif- 
ically, if this were not the case, then one would have d + (A, ft \ B') < T and 
d~(VL \ B',A) > 99T, which contradicts the bound of n(A) > 0.01 using Proposi- 
tion ll.4l We now claim that tt(B) > 0.96. Indeed, if on the contrary ir(B'\B) were 
greater than 0.02, then one would obtain E x [t a ] < E x [t b >\b] + 98.9T < 99.9T, a 
contradiction. 
Now, define 

C = {y E il : E y [t a ] > 99.8T}. 

See Figure [3] We claim that ir(C) < 0.01. Indeed, if ir(C) were greater than 0.01, 
then the set A U C would have stationary measure at least 0.02, so that d + (il, A U 
C) < T. And we would then obtain from Lemma l4~2l that d+(B,C) < 14T. On 
the other hand, d~(C,B) > 0.8T (otherwise, d~(C, A) < d-(C,B) + d+(B,A) < 
0.8T + 99T, which contradicts the definition of C). And so, by Proposition ll.4[ 
ir(B) < 14T/14.8T = 70/74 < 0.96, a contradiction. Thus we have 7r(0\C) > 0.99 
and the inequality 99.9T = E x [t a ] < E :c [r f2 \ c ] + 99. 8T implies that E x [t q \ c ] > 
0.1T. Therefore T(0.99) > 0.1T, as required. □ 

Acknowledgements. We are grateful to Yuval Peres and Perla Sousi for their kind 
permission to include their proof of Proposition [L4l We also thank the anonymous 
referee for helpful comments and corrections. 
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Appendix A. A lemma for the proof of Proposition 11.41 



For completeness, we include here a proof of an assertion used in the proof of 
Proposition [T^] This is similar to the proof of Proposition 1.14 of [5]. 

Lemma A.l. Suppose we are given an irreducible Markov chain X with finite state 
space fi and stationary distribution tt. Let fi be a distribution on fi and r be an 
a.s. positive stopping time such that P M (X T = x) = fJ,(x) for any x £ fi. Then for 
any S C fi the expected time X spends in S up to time r starting from fi equals 
tt(S)E u [t]. 

Proof. For each x £ fi, define tt(x) as the expected time X spends at x up to time 
r when started from fi, i.e. 



tt(x) = E M 



t<T} 



,t=0 



We shall prove that (tt P)(x) — tt(x) for all x £ fi, which implies that tt is a 
multiple of the (unique) stationary distribution tt. The lemma then follows from 
J2xen k{ x ) — Mi so t na t k(x) — tt(x) E^ [t] for all x £ fi. 
Let us now fix x £ fi and compute (tt P)(x): 



(nP)(x) = J2^ 
yen 



E 1 {X t =y,t<T 



P ^ = EE P m (Xt=y,t<r) P y . 
t=o yen 



since all terms are non-negative. By the Markov property, each term of the double 
sum equals P M (X t — y, X t +\ = x,t < r), and resolving the inner sum gives 



(tt P)(x) = E P m ( X t+i =x,t<r). 



t=o 

Now we split each term in the last summation into two parts as follows: 

P M (X t +1 = »,t < 7") = P„ (X t+ 1 =X,t + l<T)+V^ (X t+1 =x,t + l = r). 

Summing the first part over t gives 

oo 

J2 P M ( x t+i =x,t + l<r)= n(x) - P M (X =x,r>0) 

= tt(x) - P M (X =x) = tt(x) - n(x), 
where the second equality uses the fact that r > a.s. Summing the second part, 

oo 

]T P M (X t+1 = X ,t + l = T)=V fl (X T =x) = ti{x), 
t=0 

where we have used t > a.s. for the first equality and the assumption on \i for 
the second. It follows that (jt P)(x) = tt(x) — fi(x) + n(x) = tt(x), as desired. □ 
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